1. Field
The present disclosure pertains to methods and apparatus for processing images. More specifically, it employs systems and software to detect and identify specific human movements or postures, which are predictors of certain activities so that interested third parties may be notified, with a preferred embodiment operating as an observation/supervision device for patients in bed who have been identified as fall risk.
2. Statement of the Problem
A variety of commercially available sensor types may be used to monitor space and report on a wide range of parameters. The sensor types are associated with different technologies, each exhibiting its own set of advantages and disadvantages. There is a need to improve sensor performance, accuracy and the ability to assess parameters of interest within the monitored space.
The most widely used sensors are of a low cost design. These sensors are crude by modern information processing standards. The currently available sensors for motion, for example, are unable to provide any information regarding detected motion and therefore, trigger for any motion that occurs in the field of view, regardless of the source of motion. By way of illustration, when a sensor is installed to turn on lights in a driveway when a car pulls up, the motion detector problematically activates for passing dogs that walk by, trees moving in a breeze, and even for an optical heat gradient created by the rising sun. These latter occurrences are all categorized as false-positive events, since the objective is selective activation for an approaching car. These sensors correctly detect an intrusion so the system is operating with sufficient selectivity to detect an intrusion event; however, specificity is poor because the system cannot discriminate to ascertain the nature of the intrusion. One solution to avoid such problems is to limit the monitored space. For example, one manufacturer of a driveway monitor device states in the product specification that the “signal should be installed at a height of 4 feet to avoid feedback interference errors from the ground.” This solution is less than satisfactory because it fails to accurately detect intrusion events occurring below a certain height because of ground interference.
Current low cost sensors have poor specificity because there is insufficient resolution to discriminate between different objects or events that fall within their field of view. This inability to detect and respond appropriately for a detected object or event is directly responsible for increased cost of support and may even endangers lives. In one example, a small rodent running across the floor can set off a motion detector, triggering an intruder alarm and causing a police response. Some manufacturers offer ‘pet immune’ detectors, so an alarm is claimed to not trigger by the homeowner's dog for home security systems. These systems do not recognize pets. They are quantifying the heat mass of the object using an infrared sensor and, based on some threshold value, signal an alarm accordingly. Obviously, if the dog runs around the house excitedly because it hears the garbage truck outside, the heat mass will elevate possibly triggering an alarm. Furthermore, there is no ability to discriminate between a dog and a small child.
Nor is it the case that higher-end systems solve these problems in a satisfactory manner. The prior art contains a variety of programming techniques related to the processing of digital video images. Considerable prior art surrounds the processing of video images in an effort to recognize objects or events in the monitored or recorded footage. Applications are primarily intended for surveillance or security purposes. These systems use video cameras to capture video footage in the field of view and transmit the video signal, either through a cable interconnect or wirelessly, to a remote processing station where the video footage is processed. These video systems require high data bandwidth and considerable infrastructure to support the transmission of video signals to remote processing stations. These needs greatly increase the cost of deployment and operation. Also, the image analysis is performed post-event at a central processing station, which typically supports multiple video cameras. The system requirement for central processing greatly increases the required processing power of the central station, the complexity of its operation and the process is analyzing history, rather than real-time. The complexity of video systems is further increased to satisfy one of the primary purposes, which is preservation of records for future reference. Furthermore, video cameras are expensive and complex devices, which lower their reliability.
Digital imaging systems have become very inexpensive to produce. As a result, image creation is a standard feature on most modern handheld communication devices, e.g., cell phones and PDAs. Image creation technology has been commoditized to greatly reduce the cost to create a digital image. These commodity components provide a low cost, stand-alone, reliable image capture hardware platform.
Recently, systems and methods have been disclosed that have taken advantage of the commoditization of image creation technology and incorporated imaging capability into monitoring applications. U.S. Pat. No. 7,231,654, entitled “Remote Monitoring Method and Monitor Control Server” (“the '654 patent”) describes method where an image capture device is added to a remote monitoring device that is connected to monitor control server. The '654 patent essentially adds an image capture capability to a conventional motion detector used for security monitoring, where an image is captured only when the motion detector triggers and the image is transmitted to the control server without any image processing being performed by the remote monitor. The control server then formats the image for transmission to a mobile terminal, such as a PDA or cell phone, where the image may be viewed by a user.
The '654 patent repeats the problems of the prior sensor art. The disclosure does not provide any imaging processing at the remote monitor and, in fact, does not provide any image processing throughout the entire system. The image capture capability is only used to capture an image and pass it through the system using various communication methods to enable the image to be reviewed remotely. The '654 patent does not provide any improvement to the sensor art and only offers improvement to the notification methods for existing security systems.
In another example of prior art that falls short of providing any improvement to the general sensor art is disclosed in U.S. Pat. No. 6,697,104 (“the '104 patent”), entitled “Video Based System and Method for Detecting and Counting Persons Traversing an Area Being Monitored.” The '104 patent discloses a dedicated remote video device having a single function, namely, to count the number of people that enter or exit a monitored area. The data collected by the remote monitoring device must be uploaded to a remote processing station where additional processing activity is required to provide any useful value. Several prerequisites exist regarding the deployment environment that severely limits the functional value of the disclosure. The '104 patent provides some limited ability to ‘configure’ the remote monitoring device once it is installed at its deployment location to account for the environment of use however, the configuration capability is limited to the ‘area of interest,’ subdividing the field of view. Accordingly, the ‘counting’ function is limited to only a part of the captured video signal.
Using video signals, the remote monitoring device of the '104 patent processes frames of video but never actually recognizes the presence of a person. There is merely recognition of artifacts of “what might be a person.” For example, if a dog were to pass through the field of view, the system may or may not count the dog as person. This happens because the '104 patent relies upon on a static background image. There is subtraction of this background to identify a patterned characteristic change in the background that is assumed to represent a person; however, this is not a process that recognizes the object itself. Only an artifact of the image is observed as a change in the background. Furthermore, the '104 patent registers person counts in some internal counters that must be uploaded to another computer system where the counts must be processed and formulated into a useable format for a subscription based customer model. The remote monitoring device does not produce output that contains any actionable information. The single, dedicated function of the remote monitoring device cannot be reprogrammed, and its inability to receive ‘count’ objects other than people is limiting.
U.S. Pat. No. 7,190,259 (“the '259 patent”) discloses a lens arrangement to create a 360 degree view for a video imaging component. The system displays the captured image on a screen. The lens is deployed in the exterior rearview mirrors of an automobile. The image is displayed for driver viewing. The omni-directional vision sensor merely delivers output for presentation of the image and does not process the image.
U.S. Pat. No. 7,414,647 (“the '647 patent”) describes another lens arrangement to create a 360 degree view for a video imaging component. Again the system displays the image on a remote terminal screen. The image capture and processing device has limited processing to format the image data into either panoramic or perspective views for display purposes. A mobile body detecting section is disclosed for detecting a moving body in the field of view. The primary application disclosed is for deployment in an ATM to capture images of ATM users. A ‘communication section’ telecommunicates with an external terminal device. The data can be transmitted either wirelessly or via cable interconnect but again, its limited strictly to image data. There is no discussion anywhere in the disclosure of the image capture and processing device receiving any information or of the communication section being used to receive data.
U.S. Pat. No. 7,200,246 (“the '246 patent”) discloses an imaging system and method directed toward industrial equipment safety by monitoring an ‘area of interest’ and detecting an object entering the area. The image capture and processing device contains image processing algorithms for object detection, an output signal and the ability to connect a PC to the image capture and processing device for configuration. However, the disclosed capability all relates to an ‘area of interest’ in the manner of a motion detector and do not address the selective identification of ‘objects of interest’, nor are any image processing functions or algorithms transferred to image capture and processing device through the PC connection.
The ‘area of interest’ is monitored by the image capture and processing device employing a defined border area. The image processing relies on the use of a reference image, where sampled images are subtracted from the reference image and changes in pixel pigmentation are used to determine if an object has ‘breached’ the border of the monitored area of interest. While the disclosure includes discussion of establishing a ‘threshold’ for the number of pixels required to trigger an output signal, even referencing a “hand”, there is no attempt by the '246 patent disclosure to recognize the object breaching the border.
Human falls are the leading cause of injury for persons over the age of 65 and can occur in any environment, including hospitals, long term care, retirement housing or single family dwellings. Falls are especially prevalent in health care environments where patients under care generally have reduced physical or mental capacity. The occurrence of falls in health care environment is so pronounced that the reduction of the number of injuries resulting from patient falls is one of the Joint Commission's patient safety goals. The growth in the number of older adults has led to more aggressive efforts to promote independent living and expansion in monitoring the well being of older individuals, for all possible living environments. Given the occurrence of falls, the need to monitor and report has become a critical element of society. While falls can occur during any activity, the great percentage of incidences occurs when an individual transitions from lying down or sitting.
Monitoring human activity is not limited to fall risk individuals. General monitoring of elderly who live alone is a prime activity monitoring application. Many other applications exist, for example, with prison and jail overcrowding, the image capture and processing system can be trained to look for particular movements or postures, which are characteristic of activities that may be harmful to prison personnel or inmates, specifically inmates who are placed on suicide watch. Other example applications include schools, which have become increasingly hazardous and the present instrumentality can be used to monitor children leaving designated areas or other individuals entering certain school areas and identifying suspicious articles being carried. Other applications will become apparent to those skilled in the art and this disclosure will focus on a generalized description of a bed monitor application.
There are a number of different patient bed monitoring systems and methods disclosed in the prior art. Most of the bed monitoring systems disclosed involves some contact with the patient. Patient contact requires the sensor component of the patient bed monitor system to be properly positioned or attached to the patient, which makes it subject to mis-alignment or removal by the patient that generally results in a false alarm. Another considerable drawback of system and methods described in prior art is that they are limited to reporting on the presence or absence of patient, as described in U.S. Pat. No. 6,917,293 to Beggs (Beggs '293). Reporting the absence of a patient from bed is too late as the patient is already out of bed unassisted with a 30% chance he or she has already fallen.
One disclosed bed monitoring system utilizes image analysis and purports to provide utility for fall prevention and detection. The system described in U.S. Pat. No. 5,541,934 to Fredriksson (the '934 patent), as with other devices, is primarily used for reporting that a fall has occurred. Further, the disclosed invention relies on an optical sensor given in a separate patent, where a central station is claimed for receiving images to be reviewed by an operator. Another drawback of the '934 patent is that it requires images to be saved for later analysis if some future event occurs. This specifically teaches away from a real-time image analysis system. Saving images and transmitting to a remote location raises privacy concerns for patient monitoring and makes the '934 patent unsuitable for use by health care facilities.
The '934 patent includes language whereby the invention purports to provide fall prevention by identifying a person leaving the bed to end up standing beside the bed. The need for the invention to have a person go from laying down to standing beside it in order to recognize the motion before issuing an alarm renders the invention commercially unsuitable for fall prevention. Once the person is standing, the invention provides nothing to prevent the individual from falling and any alarm is too late. This also holds true for an alarm if a person sitting on the edge of the bed as it results in reduced reaction time for a care giver to arrive before the person is standing. The fundamental drawback of the '934 patent is that the methods described with the various algorithms are only able to detect that a change has occurred between the current image and previous images. The '934 patent classifies all changes between images as motion and then attempts to measure the change at defined states (standing, lying on the floor, etc.). The limitations imposed by the '934 patent are due to the fact that the system uses “blobs”, which is a term of art in the computer science field and can be generally regarded as a collection of binary data. Blobs do not provide any internal information and can only be characterized by their size and, if an edge detection algorithm is properly applied, information regarding their shape. In the case of the '934 patent, the disclosed invention makes use of the orientation, or aspect ratio of blobs to trigger alarm conditions without any knowledge of what the blob represents. Again, the '934 patent teaches away from real-time image processing and issuing an alarm by incorporating a delay period needed to confirm the orientation of the blob. The '934 patent does not contemplate any attempts to classify the ‘type’ of motion. It is necessary to detect and identify specific type of motion or postures in order to provide a reliable predictor, or notification, that a person is about to get out of bed.
All disclosed patient monitoring prior art incorporates the requirement to delay issuing an alarm to reduce the incidence of false alarm.
The need exists for an improved system and method that provides care takers with a reliable, advanced warning or notification that a patient is about to get out of bed without assistance and that does so without delay and a very low incidence of false alarms.